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Abstract 

We study the problem of generating a shared secret key between two terminals in a joint source- 
channel setup — the sender communicates to the receiver over a discrete memoryless wiretap channel 
and additionally the terminals have access to correlated discrete memoryless source sequences. We 
establish lower and upper bounds on the secret-key capacity. These bounds coincide, establishing 
the capacity, when the underlying channel consists of independent, parallel and reversely degraded 
wiretap channels. In the lower bound, the equivocation terms of the source and channel components 
are functionally additive. The secret-key rate is maximized by optimally balancing the the source 
and channel contributions. This tradeoff is illustrated in detail for the Gaussian case where it is also 
^ shown that Gaussian codebooks achieve the capacity. When the eavesdropper also observes a source 

Q I sequence, the secret-key capacity is established when the sources and channels of the eavesdropper 

are a degraded version of the legitimate receiver. Finally the case when the terminals also have 
access to a public discussion channel is studied. We propose generating separate keys from the 
^ ■ source and channel components and establish the optimality of this approach when the when the 

l/^ ' channel outputs of the receiver and the eavesdropper are conditionally independent given the input. 

cn ■ 

OO 

' I. Introduction 

■ Many applications in cryptography require that the legitimate terminals have shared secret- 

0\ , keys, not available to unauthorized parties. Information theoretic security encompasses the 

study of source and channel coding techniques to generate secret-keys between legitimate 
terminals. In the channel coding literature, an early work in this area is the wiretap channel 
^ ■ model [19]. It consists of three terminals — one sender, one receiver and one eavesdropper. 

The sender communicates to the receiver and the eavesdropper over a discrete-memoryless 
broadcast channel. A notion of equivocation-rate — the normalized conditional entropy of the 
transmitted message given the observation at the eavesdropper, is introduced, and the tradeoff 
between information rate and equivocation rate is studied. Perfect secrecy capacity, defined 
as the maximum information rate under the constraint that the equivocation rate approaches 
the information rate asymptotically in the block length is of particular interest. Information 
transmitted at this rate can be naturally used as a shared secret-key between the sender and 
the receiver. 

In the source coding setup [1], [15], the two terminals observe correlated source sequences 
and use a public discussion channel for communication. Any information sent over this 
channel is available to an eavesdropper. The terminals generate a common secret-key that is 
concealed from the eavesdropper in the same sense as the wiretap channel — the equivocation 
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rate asymptotically equals the secret-key rate. Several multiuser extensions of this problem 
have been subsequently studied. See e.g., [5], [6]. 

Motivated by the above works, we study a problem where the legitimate terminals observe 
correlated source sequences and communicate over a wiretap channel and are required to 
generate a common secret-key. One application of this setup is sensor networks, where 
terminals measure correlated physical processes. It is natural to investigate how these mea- 
surements can be used for secrecy. In addition, the sensor nodes communicate over a wireless 
channel where an eavesdropper could hear transmission albeit through a different channel. 
Another application is secret key generation using biometric measurements [7]. During the 
registration phase, an enrollment biometric is stored into a database. To generate a secret key 
subsequently, the user is required to provide another measurement of the same biometric. This 
new measurement differs from the enrollment biometric due to factors such as measurement 
noise and hence can be modeled as a correlated signal. Again when the database is remotely 
located, the communication happens over a channel which could be wiretapped. 

The secret-key agreement scheme, [1], [15], generates a secret key only using the source 
sequences. On the other hand, the wiretap coding scheme [19] generates a secret-key by 
exploiting the structure of the underlying broadcast channel. Clearly in the present setup, we 
should consider schemes that take into account both the source and channel contributions. 
One simple approach is timesharing — for a certain fraction of time the wiretap channel is 
used as a (rate limited) transmission channel whereas for the remaining time, a wiretap code 
is used to transmit information at the secrecy capacity. However such an approach in general 
is sub-optimal. As we will see, a better approach involves simultaneously exploiting both the 
source and channel uncertainties at the eavesdropper. As our main result we present lower 
and upper bounds on the secret-key capacity. The lower bound is developed by providing a 
coding theorem that consists of a combination of a Wyner-Ziv codebook, a wiretap codebook 
and a secret-key generation codebook. Our upper and lower bounds coincide, establishing the 
secret-key-capacity, when the wiretap channel consists of parallel independent and degraded 
channels. 

We also study the case when the eavesdropper observes a source sequence correlated with 
the legitimate terminals. The secret-key capacity is established when the sources sequence 
of the eavesdropper is a degraded version of the sequence of the legitimate receiver and 
the channel of the eavesdropper is a degraded version of the channel of the legitimate 
receiver. Another variation — when a public discussion channel is available for interactive 
communication, is also discussed and the secret-key capacity is established when the channel 
output symbols of the legitimate receiver and eavesdropper are conditionally independent 
given the input. 

The problem studied in this paper also provides an operational significance for the rate- 
equivocation region of the wiretap channel. Recall that the rate-equivocation region captures 
the tradeoff between the conflicting requirements of maximizing the information rate to 
the legitimate receiver and the equivocation level at the eavesdropper [3]. To maximize 
the contribution of the correlated sources, we must operate at the Shannon capacity of the 
underlying channel. In contrast, to maximize the contribution of the wiretap channel, we 
operate at a point of maximum equivocation. In general, the optimal operating point lies in 
between these extremes. We illustrate this tradeoff in detail for the case of Gaussian sources 
and channels. 

In related work [10], [16], [20] study a setup involving sources and channels, but require 
that a source sequence be reproduced at the destination subjected to an equivocation level at 
the eavesdropper. In contrast our paper does not impose any requirement on reproduction 
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of a source sequence, but instead requires that the terminals generate a common secret 
key. A recent work, [18], considers transmitting an independent confidential message using 
correlated sources and noisy channels. This problem is different from the secret-key generation 
problem, since the secret-key, by definition, is an arbitrary function of the source sequence, 
while the message is required to be independent of the source sequences. Independently and 
concurrently of our work the authors of [17] consider the scenario of joint secret-message- 
transmission and secret-key-generation, which when specialized to the case of no secret- 
message reduces to the scenario treated in this paper. While the expression for the achievable 
rate in [17] appears consistent with the expression in this paper, the optimality claims in [17] 
are limited to the case when either the sources or the channel do not provide any secrecy. 

The rest of the paper is organized as follows. The problem of interest is formally introduced 
in section |II] and the main results of this work are summarized in section Unl Proofs of the 
lower and upper bound appear in sections HVl and [Vl respectively. The secrecy capacity for the 
case of independent parallel reversely degraded channels is provided in section |Vll The case 
when the wiretapper has access to a degraded source and observes transmission through a 
degraded channel is treated in section jVII] while section IVin] considers the case when a public 
discussion channel allows interactive communication between the sender and the receiver. The 
conclusions appear in section |Kl 



IL Problem Statement 

Fig. [T] shows the setup of interest. The sender and receiver communicate over a wiretap 
channel and have access to correlated sources. They can interact over a public-discussion 
channel. We consider two extreme scenarios: (a) the discussion channel does not exist (b) the 
discussion channel has unlimited capacity. 
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Fig. 1. Secret-key agreement over the wiretap channel with correlated sources. The sender and receiver communicate over 
a wiretap channel and have access to correlated sources. They communicate interactively over a public discussion channel 
of rate R, if it is available. 

The channel from sender to receiver and wiretapper is a discrete-memoryless-channel 
(DMC), Py,z\x{-, •!•)• The sender and intended receiver observe discrete-memoryless-multiple- 
source (DMMS) Pu,v{-, ■) of length N and communicate over n uses of the DMC. We 
separately consider the cases when no public discussion is allowed and unlimited discussion 
is allowed. 
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A. No discussion channel is available 

An (n, A^) secrecy code is defined as follows. The sender samples a random variable 
from the conditional distribution ]?mx|u^("|^")- The encoding function /„ : M.^ x U'^ — > 
maps the observed source sequence to the channel output. In addition, two key generation 
functions k = Kn{M.xM^) and / = L„(V^,3^") at the sender and the receiver are used 
for secret-key generation. A secret-key rate R is achievable with bandwidth expansion factor 
(3 if there exists a sequence of (n, (5n) codes, such that for a sequence e.„ that approaches 
zero as n ^ oo, we have (i) Pr(/c ^ I) < Sn (ii) \H{k) > R — Sn (iii)^-^(^; ■^") < ^n- The 
secret-key-capacity is the supremum of all achievalDle rates. 

For some of our results, we will also consider the case when the wiretapper observes a 
side information sequence sampled i.i.d. Pw{-)- In this case, the secrecy condition in (iii) 
above is replaced with 

-/(/c;z",i/i/^) <£„ (1) 

n 

In addition, for some of our results we will consider the special case when the wiretap 
channel consists of parallel and independent channels each of which is degraded. 
1) Parallel Channels: 

Definition 1: A product broadcast channel is one in which the M constituent subchannels 
have finite input and output alphabets, are memoryless and independent of each other, and 
are characterized by their transition probabilities 

M n 

Pr ({C2:;U=1,...,M I {xl,}m=l,...,M) = n llMym{t),Zm{t) I (2) 

m=l t=l 

where xj^ = {xm{l),Xm{'2), . . . ,Xmin)) denotes the sequence of symbols transmitted on 
subchannel m, where = |/m(2), . . . ,ym{n)) denotes the sequence of symbols ob- 

tained by the legitimate receiver on subchannel m, and where zl^ = {zm{l)., 2;m(2), . . . , Zm{n)) 
denotes the sequence of symbols received by the eavesdropper on subchannel m. 

■ 

A special class of product broadcast channels, known as the reversely degraded broadcast 
channel [8] are defined as follows. 

Definition 2: A product broadcast channel is reversely-degraded when each of the M 
constituent subchannels is degraded in a prescribed order. In particular, for each subchannel 
m, one of ^ y„ ^ or ^ z,„ holds. 

■ 

Note that in Def. [2]the order of degradation need not be the same for all subchannels, so the 
overall channel need not be degraded. We also emphasize that in any subchannel the receiver 
and eavesdropper are physically degraded. Our capacity results, however, only depend on the 
marginal distribution of receivers in each subchannel. Accordingly, our results in fact hold for 
the larger class of channels in which there is only stochastic degradation in the subchannels. 

We obtain further results when the channel is Gaussian. 

'The alphabets associated with random variables will be denoted by calligraphy letters. Random variables are denoted by 
sans-serif font, while their realizations are denoted by standard font. A length n sequence is denoted by x". 

^However, when we consider the presence of a public-discussion channel and interactive communication, the capacity 
does depend on joint distribution Py,z\x{') 
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2) Parallel Gaussian Channels and Gaussian Sources: 

Definition 3: A reversely-degraded product broadcast channel is Gaussian when it takes 
the form 

m = l,...,M (3) 

where the noise variables are all mutually independent, and nr,m ~ CJ\f{0, cr'^m) "e,m ~ 
CA/'(0, (Tg For this channel, there is also an average power constraint 



E 



■ M 

E 

m=l 



< P. 



Furthermore we assume that u and v are jointly Gaussian (scalar valued) random variables, 
and without loss of generality we assume that u ~ A/'(0, 1) and v = u + s, where s ~ A/'(0, S) 
is independent of u. 



B. Presence of a public discussion channel 

We will also consider a variation on the original setup when a public discussion channel 
is available for communication. This setup was first introduced in the pioneering works [1], 
[15] where the secret-key capacity was bounded for source and channel models. The sender 
and receiver can interactively exchange messages on the public discussion channel. 

The sender transmits symbols xi, . . .x„ at times < zi < Z2 < • • • < over the wiretap 
channel. At these times the receiver and the eavesdropper observe symbols Yi, y2, ■ ■ ■ , Yn and 
zi, Z2, . . . , z„ respectively. In the remaining times the sender and receiver exchange messages 
0t and %jjt where 1 < t < k. For convenience we let = k + 1. The eavesdropper observes 
both (pt and ipt. More formally, 

• At time the sender and receiver sample random variables and rriy respectively from 
conditional distributions Pmx|u^('l^^) Pmy|i/^("b^)- Note that ^ ^ 

rriy holds. 

• At times < t < ?i the sender generates (pt = ^timx, , fp^~^) and the receiver 
generates tpt = ^t('T7y, v^, 0*^^). These messages are exchanged over the public channel. 

• At times ij, I < j < n, the sender generates Xj = Xj{m^, u^,4'^^~^) and sends it over 
the channel. The receiver and eavesdropper observe yj ad zj respectively. For these times 

we set (pi^ = ipi^ = 0. 

• For times ij < t < ij^i, where I < j < n, the sender and receiver compute 0t = 
^t{m^, ,ip^~^) and ijjt = \l/j(my, i/^, y-', respectively and exchange them over 
the public channel. 

• At time k + 1, the sender and receiver compute k = Kn{m^, u^,ip'') and the receiver 
computes / = Ln{my, \/^,y",0'^). 

We require that for some sequence e„ that vanishes as n ^ oo, Pr(/c ^ I) < and 

-J(/c;z^^^0'=) <5„. (4) 
n 

111. Statement of Main Results 



It is convenient to define the following quantities which will be used in the sequel. Suppose 
that t is a random variable such that t ^ u v, and a and b are random variables such that 
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b ^ a ^ X ^ {y,z) holds and J(y; b) < I{z; b). Furthermore define 

i?ch = /(a;y), (5a) 

R:^ = I{a-y\b)-I{a-z\b) (5b) 

Rs = I{t-v), (5c) 

R^, = I{t-u)-I{t-v). (5d) 

< = /(x;y |z). (5e) 

Rt^ = I{x;y), (5f) 

We establish the following lower and upper bounds on the secret key rate in Section |IV] 
and |V] respectively. 

Lemma 1: A lower bound on the secret-key rate is given by 

i?key =/5^s + i?e"q, (6) 

where the random variables f , a and b defined above additionally satisfy the condition 

/5i?wz < -Rch (7) 

and the quantities R^, R~^ and -Rch are defined in (l5dl) . (l5cl) . (l5bl) and (l5al) respectively. 

■ 

Lemma 2: An upper bound on the secret-key rate is given by, 

^key = sup + (8) 

{(x,0} 

where the supremum is over all distributions over the random variables (x, t) that satisfy 
t ^ u ^ V , the cardinality of t is at-most the cardinality of u plus one, and 

/5i?wz < i?+h- (9) 

The quantities i?s, R^^, i?^ and i?^^ are defined in (l5cl) . (l5dl) . (l5el) and ([SB respectively. 
Furthermore, it suffices to consider only those distributions where (x, t) are independent. 



A. Reversely degraded parallel independent channels 

The bounds in Lemmas [T] and [21 coincide for the case of reversely degraded channels as 
shown in section IVI-AI and stated in the following theorem. 

Theorem 1: The secret-key-capacity for the reversely degraded parallel independent chan- 
nels in Def. [2l is given by 



Ckcy = max <^ t) + ^ /(x^; yi\zi) \ , (10) 

{(xi,...,XM,t)} I ^ J 

where the random variables (xi, . . . ,xm, t) are mutually independent, t — > tv — i/, and 

M 

J2Hx^■,y^)>P{Hu■,t)-Iiv■t)} (11) 

i=l 

Furthermore, the cardinality of t obeys the same bounds as in Lemma [2l 
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Fig. 2. An example of independent parallel and reversely degraded Gaussian channels. On the first channel, the eavesdropper 
channel is noisier than the legitimate receiver's channel while on the second channel the order of degradation is reversed. 

B. Gaussian Channels and Sources 

For the case of Gaussian sources and Gaussian channels, the secret-key capacity can be 
achieved by Gaussian codebooks as established in section IVI-BI and stated below. 

Corollary 1: The secret-key capacity for the case of Gaussian parallel channels and Gaus- 
sian sources in subsection III-A.2I is obtained by optimizing (flOl ) and (fTTI) over independent 
Gaussian distributions i.e., by selecting Xj ~ A/'(0, Pi) and u = t + d, for some d ~ A/'(0, D), 
independent of t and ^"^^ Pi < P, Pi > 0, and < D < 1. 

where D, Pi, . . . , Pm also satisfy the following relation: 



C. Remarks 

1) Note that the secret-key capacity expression (flOl) exploits both the source and channel 
uncertainties at the wiretapper. By setting either uncertainty to zero, one can recover 
known results. When I{u; v) = 0, i.e., there is no secrecy from the source, the secret- 
key-rate equals the wiretap capacity [19]. If /(x; y |z) = 0, i.e., there is no secrecy from 
the channel, then our result essentially reduces to the result by Csiszar and Narayan [5], 
that consider the case when the channel is a noiseless bit-pipe with finite rate. 

2) In general, the setup of wiretap channel involves a tradeoff between information rate 
and equivocation. The secret-key generation setup provides an operational significance 
to this tradeoff. Note that the capacity expression (flOl) in Theorem [T] involves two terms. 
The first term /3/(f; v) is the contribution from the correlated sources. In general, this 
quantity increases by increasing the information rate /(x;y) as seen from (fTTI) . The 
second term, /(x; y|z) is the equivocation term and increasing this term, often comes at 
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Fig. 3. Tradeoff inherent in the secret-key-capacity formulation. The solid curve is the secret-key-rate, which is the sum 
of the two other curves. The dotted curve represents the source equivocation, while the dashed curve represents the channel 
equivocation J18t . The secret-key-capacity is obtained at a point between the maximum equivocation and maximum rate. 

the expense of the information rate. Maximizing the secret-key rate, involves operating 
on a certain intermediate point on the rate-equivocation tradeoff curve as illustrated by 
an example below. 

Consider a pair of Gaussian parallel channels, 



(14) 



yi = aix + rir^i, zi = bix + ne,i 

y2 = aax + n^,2, ^2 = y2 

where ai = 1, 02 = 2, and bi = 0.5. Furthermore, u ~ A/'(0, 1) and v = u + s, where 
s ~ A/'(0, 1) is independent of u. The noise variables are all sampled from the CJ\f{0, 1) 
distribution and appropriately correlated so that the users are degraded on each channel. 
A total power constraint P = 1 is selected and the bandwidth expansion factor P equals 
unity. 



From Theorem [H 



key 



Pl,P2,D 

such that, 

1 



max Req{Pi,P2 



1, 2 



log — 

2 ^ D 



- loff - 

2 ^ 1 



<-(log(l + a?Pi) 



D 
log(l 



IP2)) , 
log(l + blP^)) . 



(15) 

(16) 
(17) 
(18) 



Fig. [3] illustrates the (fundamental) tradeoff between rate and equivocation for this 
channel, which is obtained as we vary power allocation between the two sub-channels. 
We also present the function i?src = lit; v) which monotonically increases with the rate, 
since larger the rate, smaller is the distortion in the source quantization. The optimal 
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point of operation is between the point of maximum equivocation and maximum rate 
as indicated by the maximum of the solid line in Fig. [3l This corresponds to a power 
allocation (Pi, P2) ~ (0.29,0.71) and the maximum value is -Rkcy ~ 0.6719. 

D. Side information at the wiretapper 

So far, we have focussed on the case when there is no side information at the wiretapper. 
This assumption is valid for certain application such as biometrics, when the correlated sources 
constitute successive measurements of a person's biometric. In other applications, such as 
sensor networks, it is more realistic to assume that the wiretapper also has access to a side 
information sequence. 

We consider the setup described in Fig. [H but with a modification that the wiretapper 
observes a source sequence w^, obtained by N— independent samples of a random variable 
w. In this case the secrecy condition takes the form in ([T]). We only consider the case when 
the sources and channels satisfy a degradedness condition. 

Theorem 2: Suppose that the random variables (tv, v, w) satisfy the degradedness condition 
u ^ V ^ w and the broadcast channel is also degraded i.e., x ^ y ^ z. Then, the secret- 
key-capacity is given by 

Ckcy = max{/3(/(t; v) - I{t; w)) + I {x; y\z)} , (19) 

where the maximization is over all random variables (t, x) that are mutually independent, 
t —>■ u —>■ V —>■ w and 

I{x;y)>/3{I{u;t)-I{v;t)) (20) 

holds. Furthermore, it suffices to optimize over random variables t whose cardinality does 
not exceed that of u plus two. 



E. Secret-key capacity with a public discussion channel 

When public interactive communication is allowed as described in section III-B[ we have 
the following upper bound on the secret-key capacity. 

Theorem 3: An upper bound on the secret-key capacity for source-channel setup with a 
public discussion channel is 

Ckcy < max/(x;y|z) + /3J(u; 1/). (21) 

Px 

The upper bound is tight when channel satisfies either x^y^z or y^x^z. 

■ 

The presence of a public discussion channels allows us to decouple the source and channel 
codebooks. We generate two separate keys — one from the source component using a Slepian- 
Wolf codebook and one from the channel component using the key-agreement protocol 
described in [1], [15]. 

The upper bound expression (|2TI) in Theorem |3] is established using techniques similar to 
the proof of the upper bound on the secret-key rate for the channel model [1, Theorem 3]. 
A derivation is provided in section IVIIII 

Fig. |4] illustrates the contribution of source and channel coding components for the case of 
Gaussian parallel channels (fT4l) consisting of (physically) degraded component channels. The 
term /(tv; v) is independent of the channel coding rate, and is shown by the horizontal line. 
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Fig. 4. Secret-key-rate in the presence of a public discussion channel in the Gaussian example l ll4t . The solid curve is the 
secret-key-rate, which is the sum of the two other curves. The horizontal line is the key rate from the source components. 
Regardless of the channel rate, the rate is 0.5 bits/symbol. The dashed-dotted curve is the key-rate using the channel J(x; yjz). 

The channel equivocation rate /(x; y |z) is maximized at the secrecy capacity. The overall key 
rate is the sum of the two components. Note that unlike Fig. [3l there is no inherent tradeoff 
between source and channel coding contributions in the presence of public discussion channel 
and the design of source and channel codebooks is decoupled. 



IV. Achievability: Coding Theorem 

We demonstrate the coding theorem in the special case when a = x and 6 = in Lemma [B 
Accordingly we have that ( |5al) and (|5b1 ) reduce to 

i?ch = /(x;y) (22a) 

R-^ = I{x;y)-I{x-z) (22b) 

The more general case, can be incorporated by introducing an auxiliary channel a — x and 
superposition coding [4] as outlined in Appendix HI Furthermore, in our discussion below we 
will assume that the distributions pt\u and px are selected such that, for a sufficiently small 
but fixed (5 > 0, we have 

/?i?wz = Rch - 36. (23) 

We note that the optimization over the joint distributions in Lemma \T\ is over the region 
PRwz < -Rch- If the joint distributions satisfy that PR^z = ce{Rch — 35) for some a < 1, one 
can use the code construction below for a bock-length an and then transmit an independent 
message at rate R^^ using a perfect-secrecy wiretap-code. This provides a rate of 

as required. 
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Fig. 5. Source-Channel Code Design for secret-key distillation problem. The source sequence u is mapped to a codeword 
in a Wyner-Ziv codebook. This codeword determines the secret-key via the secret-key codebook. The bin index of the 
codeword constitutes a message in the wiretap codebook. 



A. Codebook Construction 

Our codebook construction is as shown in the Fig. [51 

An intuition behind the codebook construction is first described. The wiretap channel 
carries an ambiguity of 2"^^*^^'^l'^^^^*^^'^l'^)j^ at the eavesdropper for each transmitted message. 
Furthermore, each message only reveals the bin index. Hence it carries an additional am- 
biguity of 2^^^^'^^ codeword sequences. Combining these two effects the total ambiguity is 

2n{/{a;y|t)-/(a;z|fa)+/3/(>/;t)} jj^^^ ^ sccrct-kcy Can bc produccd at the rate /(a; y\b)- 1 {a; z\ b) + 

(31 {v; t). This heuristic intuition is made precise below. 

The coding scheme consists of three codebooks: Wyner-Ziv codebook, secret-key code- 
book and a wiretap codebook that are constructed via a random coding construction. In our 
discussion below we will be using the notion of strong typicality. Given a random variable t, 
the set of all sequences of length N and type that coincides with the distribution pt is denoted 
by . The set of all sequences whose empirical type is in an er-shell of pt is denoted by T/^. 
The set of jointly typical sequences are defined in an analogous manner. Given a sequence 
of type Tj^, the set of all sequences that have a joint type of Pu,v{) is denoted by 
T^^{u^). We will be using the following properties of typical sequences 

|T/^,| = exp(iV(i/(t) + o,(l))) (24a) 
Pr(t^ = t^) = exp{-N{H{t) + o,(l))), V G T,^, (24b) 
Pr(t^eT,^J> 1-0.(1), ' (24c) 

where 0.(1) is a term that approaches zero as ^ oo and e ^ 0. 
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Fig. 6. Equivocation at the eavesdropper tlirough tire source-clrannel codebook. The channel codebook induces an ambiguity 
of 2"'^'^^'-''l'''^^'^'^l'''' among the codeword sequences a" when the decoder observes z". Each sequence a" only reveals the 

bin index of the Wyner-Ziv codeword. In induces an ambiguity of 2^^'''"^ at the eavesdropper, resulting in a total ambiguity 

of 2-^('3I(t;v)+I(s;y\b))-I(3:z\b) ^ 



For fixed, but sufficiently small constants 6 > and t] = 6/ 13 > 0, let, 

Mwz = exp(A^(i?s-^)) (25a) 

iVwz = exp{N{R^, + 2r])) (25b) 

MsK = exp(n(/(x;z) -5)) (25c) 

NsK = exp{n{f3R, + - 5) ) (25d) 
Substituting dSaJ-dSd]) and ^ into (l25al) - (l25dl) we have that 

iVtot = MsK ■ NsK = Mwz ■ iVwz = exp{N{I{t; u) + r/)) (26) 

We construct the Wyner-Ziv and secret-key codebooks as follows. Randomly and indepen- 
dently select iVtot sequences from the set of t— typical sequences . Denote this set T. 
Randomly and independently partition this set into the following codebook^: 

• Wyner-Ziv codebook with A'wz bins consisting of Afwz sequences. The j^^ sequence in 
bin i is denoted by tj^vvz- 

• Secret-key codebook with A^^sk bins consisting of Msk sequences. The j**^ sequence in 
bin i is denoted by fj^sK- 

We define two functions <l>wz : T — ^ {1, • • • , ^wz} and $sk : T {1, • • • , ^sk} as 
follows. 

Definition 4: Given a codeword sequence , define two mappings 

1) ^^wzit^) = I, if 3j G [1, Mwz], such that = t^^z- 

2) $sK(t^) = I, if 3j G [1, Msk] such that = t^JsK- 

■ 

The channel codebook consists of AVz = exp(?7,(i?ch — S)) sequences x" uniformly and 
independently selected from the set of x— typical sequences T". The channel encoding func- 
tion maps message i into the sequence x", i.e., $ch : {li-'-^-^wz} is defined as 

'As will be apparent in the analysis, the only pairwise independence is required between the codebooks i.e., Vt^, € T, 
Pr ($wz(t^) = $wz(t^)|$SK(t^) = $SK(t^)) = Pr ($wz(t^) = $wz(t^)) = ^ 
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B. Encoding 

Given a source sequence u^, the encoder produces a secret-key k and a transmit sequence 
as shown in Fig. |5l 

• Find a sequence E T such that (u^,t^) G T^^. Let £i be the even that no such 
exists. 

. Compute = $wz(^^) and k = $sk(^^)- Declare /c as the secret-key. 

• Compute = ^chi<P), and transmit this sequence over n— uses of the DMC. 

C. Decoding 

The main steps of decoding at the legitimate receiver are shown in Fig. [5] and described 
below. 

• Given a received sequence y", the sender looks for a unique index i such that (x", y") G 
T^ g. An error event £^2 happens if is not the transmitted codeword. 

• Given the observed source sequence v^, the decoder then searches for a unique index 
j e [l,Mwz] such that (^ij^wz'^^) ^ '^t^,e- error event £^3 is declared if a unique 
index does not exist. 

• The decoder computes k = $sK(^ij,wz) ^^'^ declares k as the secret key. 

D. Error Probability Analysis 

The error event of interest is £ = {k ^ k}. We argue that selecting n — 00 leads to 



Pr(^) ^ 0. 

In particular, note that Pr(^) = Pr(^i U ^2 U ^3) < Pr(^i) + Pr(^2) + Pr(^3)- We argue 



that each of the terms vanishes with n ^ 00. 

Recall that £1 is the event that the encoder does not find a sequence in T typical with u^. 
Since T has exp(iV(J(u; t) + r/)) sequences randomly and uniformly selected from the set 
T/^, we have that Pr(£i) ^ 0. 

Since the number of channel codewords equals Ny^z = exp(n(/(x; y) — 5)), and the 
codewords are selected uniformly at random from the set T"^, the error event Pr(£^2) 0. 

Finally, since the number of sequences in each bin satisfies Mwz = exp(A^(J(t; v) — 77)), 
joint typical decoding guarantees that Pr(£^3) 0. 

E. Secrecy Analysis 

In this section, that for the coding scheme discussed above, the equivocation at the eaves- 
dropper is close (in an asymptotic sense) to -Rkcy 

First we establish some uniformity properties which will be used in the subsequent analysis. 

1) Uniformity Properties: In our code construction $wz satisfies some useful properties 
which will be used in the sequel. 

Lemma 3: The random variable <l>wz in Def. |4] satisfies the following relations 



-i/($wz) = pRwz + 0^(1) 
n 

ii7(t^|<l>wz) =/?/(t; v) + o^{l) 
ii7($wz|z") = /(x; y) - /(x; z) + o,(l) 



(27b) 



(27a) 



(27c) 
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where o,(l) vanishes to zero as we take 77 — * and N ^ 00 for each 77. 

Proof: Relations (I27al) and (I27bl) are established below by using the properties of typical 
sequences (c.f. (I24al) - (l24cl) '). Relation (I27cl) follows from the secrecy analysis of the channel 
codebook when the message is $wz- The details can be found in e.g., [19]. 

To establish (I27al) . define the function Fwz : ^ {1, • • • , ^wz} to identify the position 
of the sequence G T in a given bin i.e., ^wzitij^wz) = J ^^^^ that. 



Pr(^wz=J,<fwz = ^)< Yl P^^""^) ™ 

J2 expi-NiH{u)+o,{l))) (29) 

= exp{N{H{u\t) + 0,(1))) exp{-N{H{u) + o,(l))) (30) 

= exp(-iV(J(f;tv)+o,(l))) (31) 



where (l28l) follows from the construction of the joint-typicality encoder, (|29l) from (I24bl) 
and (l30l) from (|24al) . Marginalizing (l28l) . we have that 

A/wz 

Pr($wz = = XI P^(rwz = j, $wz = 

<Mwzexp(-iV(/(t; u) + o,(l))) 

= exp(-iV(/(t; ty)-/(t; i/) + o,(l))) 

= exp(-iV(i?wz + 0,(1))) (32) 

Eq. (127 al) follows from (|32l) and the continuity of the entropy function. Furthermore, we 
have from (|3TI) that 

^if (<l>wz, Twz) = lit; u) + 0,(1). (33) 
The relation (I27bl) follows by substituting (I27al) . since 

li/(t^|$wz) = ^i/(rwz|*wz) = ^i/(rwz,$wz) - ^i^($wz) = /(t; >/) + o,(i). 

(34) 

■ 

Lemma 4: The construction of the secret-key codebook and Wyner-Ziv codebook is such 



that the eavesdropper can decode the sequence t if it is revealed the secret-key $sk = k m 
addition to its observed sequence z". In particular 

-if(t^|z",/c) = o,(l). (35) 
n 

Proof: We show that there exists a decoding function g : x {1, 2, . . . , iVsK} T 



that such that Pr(t^ 7^ ^'(■^"7 /c)) ^ as n — 00. In particular, the decoding function (?(■,•) 
searches for the sequences in the bin associated with k in the secret-key codebook, whose 
bin-index in the Wyner-Ziv codebook maps to a sequence x" jointly typical with the received 
sequence z'\ More formally, 

• Given z", the decoder constructs a the set of indices = {i : (xf , z") G T^_e}. 



15 



• Given k, the decoder constructs a set of sequences, 5 = 1 sk • ^wz(i'^^sK ) e Jx,i < j <MsK,}. 

• If S contains a unique sequence , it is declared to be the required sequence. An error 
event is defined as 

= { 3j, 1 < J < MsK , $wz ( t,";,, sk) e Jx, Jo } , (36) 
where jo is the index of the sequence in bin k of the secret-key codebook, i.e., 

i-N _ i-N 
'^kjo,SK — ■ 

It suffices to show that Pr(j7') ^ as n ^ oo. 
We begin by defining the following events: 

• The event that the sequence ^ S, which is equivalent to 

From (I24cl) we have that Pr(Jo) = o^(l)- 

• For each j = 1, 2, . . . Msk, j 7^ jo the event Jj that the sequence t^gj^ G S, 

J, = {'fwz(t5,sK)eXx}. 

• For each j = 1, 2, . . . MsK,j 7^ jo, define the collision event that t^^sK ^nd t^^^sK belong 
to the same bins in the in the Wyner-Ziv codebook 

J'colj = {*Wz(tfcJ,SK) = *Wz(t^(,,SK)} • 

Now we upper bound the error probability in terms of these events. 

Pr(J) <Pr(J|Jo^) + Pr(Jo) 

< J2 P<Jj\Jo)+o,{l), (37) 



Now observe that 



PriJ^Uo) = Pr(J,- n J:,,j\Jo') + Pr(J,- n JcoiM (38) 

< Pr(J, n XiJ^o) + PrlJ-coiJJo') 

< Ft{J,\J,^ n + Pr(J-coi,j|Jo'). (39) 

We bound each of the two terms in (|39| ). The first term is conditioned on the event that the 
sequences gj^ and tj^j^ g^^ are assigned to independent bins in the Wyner-Ziv codebook. 
This event is equivalent to the event that a randomly selected sequence belongs to the 
typical set X^. The error event is bounded as [2] 



Pr( n J^,,^^) < exp(-n(J(x; z) - 3e)). (40) 

To upper bound the second term, 

Pr(J,|Jo') = Pr(J,) (41) 
= exp(-n(/3i?wz + 26)) (42) 
= exp{-n{I{x;y)-6)) (43) 

where (|4TI) follows from the fact the event J'q is due to the atypical channel behavior and 
is independent of the random partitioning event that induces Jj, (|42|) follows from the fact 
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that each sequence is independently assigned to one of exp{n(/9-Rwz + 25)} bins in the code 
construction and (l43l) follows via relation (l23l) . 
Substituting (gS]) and (gO]) into (l39l), we have 



Pr(j;|Jo') < exp(-n(J(x;z) - Se)) + exp(-n(/(x; y) - 5)) 

< exp(-ra(/(x; z) - 4£:)), n > tt-q, (44) 

where we use the fact that /(x; y) > J(x; z) in the last step so that the required uq exists. 
Finally substituting (|44l) into (l37l) and using relation (I25cl) for Msk, we have that 

Fr{J) < exp{-n{5 - Ae)) + o,(l), (45) 

which vanishes with n, whenever the decoding function selects e < S/A. I 

2) Equivocation Analysis: It remains to show that the equivocation rate at the eavesdropper 
approaches the secret-key rate as n — > oo, which we do below. 

i/(/c|z") = H{k, t^|z") - H{t^\z'\ k) 

= H{t^\z'') - H{t^\z'',k) (46) 

= H{t^, $wz|z") - iJ(t^|z", k) (47) 
= i/(t^|$wz, z") + i/($wz|z") - //(t^|z", /c) 

= i/(t^|$wz) + i^($wz|z") - i/(t^|z", k), (48) 

= 1/) + n{I{x; y) - /(x; z)} + no,(l) (49) 

= n(i?key + o„(l)), (50) 

where (l46l) and (l47l) follow from the fact that $wz is a deterministic function of and (|48]) 
follows from the fact that $wz z" holds for our code construction, and (|49l) step 

follows from (|27b|) and (I27c|) in Lemma [3] and Lemma IH 

V. Proof of the Upper bound (Lemma [2]) 

Given a sequence of (n, N) codes that achieve a secret-key -rate i?kcy, there exists a sequence 
En, such that ^ as n ^ oo, and 

-i7(/c|y", < (51a) 

n 

-if(/c|z") > -H(k) - En. (51b) 

n n 

We can now upper bound the rate i?kcy as follows. 

7T,i?key = H{k) 

= i/(/c|y",(/^) + /(/c;y",i/^) 

< UEn + /(/c; y", v"") - I{k; z") + I{k; z") (52) 
<2n£„ + /(/c;y",^^)-/(/c;z") (53) 
= 2nEn + /(/c; y") - I{k; z") + J(/c; i/^|y") 

< 2n£„ + /(/c; y") - /(/c; z") + I{k, y"; (54) 

where (|52l) and (l53l) follow from (I51al) and (I51bl) respectively. 
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Now, let J be a random variable uniformly distributed over the set {1,2,...,A^} and 
independent of everything else. Let ti = (/f, y", vf^^, u\~^) and t = (/f, y*^, i/j^i, u(~^, J), and 
vj be a random variable that conditioned on J = z has the distribution of p^,- . Note that since 
is memoryless, vj is independent of J and has the same marginal distribution as v. Also 
note that t ^ uj ^ vj holds. 



I{k,y-;v^) = Y,Hk,y'';v.\vt+,) 

4 = 1 

N 

<5^/(/c,y",<^,;u,) 



i=l 
N 



<^J(/c,y",ur^„ai-V, 



i=l 

= NI{k,y\v:}^„u(-'-Vj\J) 

= NI{k,y^, v]^,, u(-\ J; vj) - /(J; vj) 

= NI{t; v) (55) 

where (l55l) follows from the fact that vj is independent of J and has the same marginal 
distribution as v. 

Next, we upper bound I(k;y'^) — I(k;z^) as below. Let p^^ denote the channel input 
distribution at time i and let ^. denote the corresponding output distribution. Let p^ = 
n X]r=i Pxi Py ^'^^ Pz be defined similarly. 

/(/c;y")-/(/c;z")</(/c;y'^|z") 

</(x";y"|z") (56) 

n 

<Y,Hx^■,y^\Zi) (57) 

i=l 

<n/(x;y|z), (58) 

where (|56l ) follows from the Markov condition /c ^ x" ^ (y", z") and (1571) follows from 
the fact that the channel is memoryless and (|58l) follows from Jensen's inequality since the 
term /(x;y|z) is concave in the distribution p^ (see e.g., [13, Appendix-I]). 
Combining (|58l) and (l55l) we have that 

i?key </(x;y|z)+/5J(i/;t), (59) 

thus establishing the first half of the condition in Lemma [21 It remains to show that the 
condition 

P{I{t- u)-I{t- v)}<I{x-y) 
is also satisfied. Since tv^ x" — > y" holds, we have that 

n/(x;y) > /(x";y") (60) 

>/(iv^;y") (61) 
> /(u^; y", /c) - /(i/^; y", /c) - (62) 
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where the last inequality holds, since 

/(u^; /c|y") - /(^^; y", k) = -/(u^; y") + /(u^; /c|y") - /(u^; /c|y") 

</(u^;/c|y'^)-J(>/^/c|y'^) 
= i/(/c|y", v^) - i/(/c|y", t;^) 

< nsn, 

where the last step holds via (I51al) and the fact that H{k\y", tv^) > 0. 
Continuing (l62l) . we have 

nJ(x; y) > J(ty^; y^ k) - J(>/^; y", /c) - nSn (63) 

TV 

= 5^{/(u.;y", /c, ui-Vf+i) - /(>/,; y", k, ul'vl^,)} + nSn (64) 

i=l 

= N{I{uj- y^ /c, ut'v^\,\J) - I{vj; y", /c, u(-'v]^,\J) + 

= Ar{/(tyj; t) - I{vj- t) + I{vj- J) - I{uj- J) + En} 

= N{I{u-t)-I{v-t) + en} (65) 

where (l64l) follows from the well known chain rule for difference between mutual information 
expressions (see e.g., [9]), (l65l) again follows from the fact that the random variables vj and 
uj are independent of J and have the same marginal distribution as v and u respectively. 

The cardinality bound on t is obtained via Caratheordory's theorem and will not be 
presented here. 

Finally, since the upper bound expression does not depend on the joint distribution of (t, x), 
it suffices to optimize over those distributions where (t, x) are independent. 

VI. Reversely Degraded Channels 

A. Proof of Theorem [7] 

First we show that the expression is an upper bound on the capacity. From Lemma [2l we 
have that 

Ckey < max/(x;y|z) + (31{t; v), 

where we maximize over those distributions where (x, t) are mutually independent, t ^ u ^ 
V, and 

/(x;y)>/3(/(t; u) - I{t- v)). 
For the reversely degraded parallel independent channels, note that 

AI 



i=l 
AI 

I{x;y\z) < ^I{xi;yi\2 



i=l 



with equality when (xi, . . . , xm) are mutually independent. Thus it suffices to take (xi, . . . , xm) 
to be mutually independent, which establishes that the proposed expression is an upper bound 
on the capacity. 
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For achievability, we propose a choice of auxiliary random variables (a, b) in Lemma [H 
such that the resulting expression reduces to the capacity. In particular, assume without loss 
in generality that for the first P channels we have that Xj ^ yj ^ Zj and for the remaining 
channels we have that Xj ^ Zj ^ y^. Let a = (xi, X2, . . . , xm) and b = (xp+i, . . . , xm) where 
the random variables {xj} are mutually independent. It follows from (|5al) and (|5bl) that 



M 



-Rch = ^/(xi;yi) (66) 

i=l 

P M 
^eq = ^ HXi'^ yiU^) = ^ {^U YiUi) , (67) 



1=1 1=1 



where the last equality follows since for Xj — > Zj ^ yj, we have that J(xj;yj|zj) = 0. 
Substituting in Q and (|7]) we recover the capacity expression. 

B. Gaussian Case (Corollary \1} 

For the Gaussian case we show that Gaussian codebooks achieve the capacity as in Corol- 
lary [B 

Recall that the capacity expression involves maximizing over random variables x = (xi, . . . , xj/) 
and t ^ u ^ V, 

i 

subjected to the constraint that ElJ^fLi^i] ^ P ^nd 

J2nx^■.y^)>P{nt;u)-I{t;v)}. (69) 



Let us first fix the distribution and upper bound the objective function (1681) . Let R = 
^"l^iLi HX'i'i Yi) and V = u + s, where s ~ A/'(0, S*) is independent of u. We will use the 
conditional entropy power inequality 

exp{2h{u + s\t)) > exp{2h{u\t)) + exp{2h{s)) (70) 

for any pair of random variables (t, u) independent of s. The equality happens if (u, t) are 
jointly Gaussian. 

Note that we can express (|69l) as 

R + h{v) - h{u) > h{v\t) - h{u\t) (71) 
= h{u + s\t) - h{u\t) (72) 

> - log {exp{2h{u\t)) + 2TTeS) - h{u\t) (73) 



Letting 



we have that 



2 



h{u\t)) = ^\og2neD, (74) 



^ - exp{2{R + h{v)-h{u))) - r ^'^^^ 
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Rearranging we have that 

M 



E/(^.;yO>f log (i + - iog(i + 5) 



(76) 



The term /(t; v) in the objective function (l68l) can be upper bounded as 

/(t; v) = h{v) - h{v\t) 

= h{v) - h{u + s\t) 

<h{v)-^ log(exp(2/i(u|s)) + 2neS) (77) 

where dTT]) follows by the application of the EPI dTO]) and dTS]) follows via dUl). Thus the 
objective function (l68l) can be expressed as 

i 

where D satisfies ( TTSl ). 

It remains to show that the optimal x has a Gaussian distribution. Note that the set of 
feasible distributions for x is closed and bounded and hence an optimum exists. Also if Px 
is any optimum distribution, we can increase both R and /(xj;yj|zj) by replacing px with a 
Gaussian distribution (see e.g., [14]) with the same second order moment. Since the objective 
function is increasing in both these terms, it follows that a Gaussian also maximizes the 
objective function (|68l ). 

VII. Side information at the Wiretapper 
We now provide an achievability and a converse for the capacity stated in Theorem [2] 



A. Achievability 

Our coding scheme is a natural extension of the case when w = 0. 

Since we are only considering degraded channels note that i?ch and i?^ in (|5a1 ) and (|5b1) 
are defined as 

i?eh = /(x; y) (80) 
i?,"q = /(x;y)-/(x;z) = /(x;y|z). (81) 

Furthermore, we replace Rg in (|5c] ) with 

R, = I{t;v)-I{t;w) (82) 

and the secret-key rate in Q is 

i?LB = PiHt; v) - I{t- w)} + /(x;y|z). (83) 

The construction of Wyner-Ziv codebook and wiretap codebook in Fig. [5] is as discussed 
in section ITV-Al HV-Bl and HV-Cl The Wyner-Ziv codebook consists of ^ 2^^^''''^ codeword 
sequences sampled uniformly from the set T^^ . These sequences are uniformly and randomly 
partitioned into ~ 2^^^^'^'")"^*^'^'^^^ bins so that there are ~ 2^^^'^'*') sequences in each bin. 
The bin index of a codeword sequence, $wz, forms a message for the wiretap codebook as 
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before. The construction of the secret key codebook is modified to reflect the side informa- 
tion sequence at the eavesdropper. In particular we construct the secret-key codebook with 
parameters 

MsK = exp (n(J(x; z) + (31{w- t)) - 6) (84) 
iVsK = exp + - 5)) (85) 

and -Rs is defined in (|82]) . 
B. Secrecy Analysis 

We show that the equivocation condition at the eavesdropper ^ holds for the code con- 
struction. This is equivalent to showing that 

ii7(/c|i/i/^, z") = v) - I{t- w)) + J(x; y|z) + o,{n), (86) 

which we will now do. 

We first provide an alternate expression for the left hand side in (l86l) . 

i/(/c|i/i/^,z") = iJ(/c,t^|i/i/^,z")-iJ(t^|/c,i/i/^,z") (87) 

= /7(t^|i/i/^,z")-i/(t^|/c, i/i/^,z") 

= if(t^,$wzk'^,z")-i/(t^|/c,i/.^,z") (88) 

= //(^wzk^, z") + i/(t^|$wz, i^^) - Hit'^lk, w^, z") (89) 

where (|88l) follows from the fact that $wz is a deterministic function of t^, while (f89l ) follows 
from the fact that ^ (i^^, '^'wz) z" forms a Markov chain. The right hand side in (l86l) 
is established by showing that 

-i7($wzk^,z") > /(x;y|z) + 0,(1) (90a) 
n 

-i/(t^|$wz, n/"^) = Pil{t; v) - I{t- w)) + 0,(1) (90b) 
n 

-H(t^\k, w^, z") = o„(l). (90c) 

n 

To interpret (I90al) . recall that $wz is the message to the wiretap codebook. The equivocation 
introduced by the wiretap codebook iif($wz|z") equals /(x;y|z). Eq. (|90al ) shows that if 
in addition to z", the eavesdropper has access to w^, a degraded source, the equivocation 
still does not decrease (except for a negligible amount). The intuition behind this claim is 
that since the bin index $wz is almost independent of v'^ (see Lemma \5\ below), it is also 
independent of due to the Markov condition. 

Eq. (I90bl) shows that the knowledge of reduces the list of sequences in any bin 
from exp(iV(/(t; v))) to exp{N{I{t; v) - I{t; w))), while (|90c| ) shows that for the code 
construction, the eavesdropper, if revealed the secret-key, can decode with high probability. 

To establish (l90al) . 

ii/($wzk'^,z'^) > -if(<l>wz|z", v^) (91) 

n n 

= -i7($wz|z") - -/($wz; v''\z^) 
n n 

> /(x;y|z) + o,(l) - -/(<l>wz; i/'^k"), (92) 

n 

> /(x;y|z) + o,(l) - -/(<l>wz; v""), (93) 

n 
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where dST]) follows from the fact that ^ ^ ($wz, ^"), (HI]) from Lemma [3] and (1931) 
from the fact that ^wz so that 

-/($wz; v^\z^) < -/(<l>wz; v^). (94) 
n n 

Thus we need to show the following. 
Lemma 5: 

-/($wz;>/^) <o,(l). (95) 

Proof: From Lemma [3] note that 



lif($wz) = /(t; ")-/(t; >/) + o,(l) 



and hence we need to show that 

1 

N 

as we do below. 



li/($wzk^) = /(t; u) - I{t- v) + 0,(1) 



^i/($wzk^) = ^i^($wz,t^lO - ^i^(t^k"^,$wz) 

= i^i/(t^|>/^) + o,(l) (96) 

Where (|96| ) follows since each bin has Mwz = exp (N(I{t; v) — 77)) sequences, (from stan- 
dard joint typicality arguments) we have that 

1 

iV 

Finally by substituting a = h = u and c = t and R = I{t; u) + rj, in Lemma [6] in 
Appendix HI] we have that 

^H{t^\v^)=I{t- u)-I{t; v) + o,{l). 



^i/(t^|>/^,$wz)=o,(l). (97) 



This completes the derivation of (1951 ). 
■ 

To establish (I90bl) . we again use Lemma [6] in Appendix HIl with a = w, b = u and c = t 
and R = I{t; v) — rj. Finally, to establish (|90cl) , we construct a decoder as in section HV-EI that 
searches for a sequence such that $wz(tfcj) G 1^ and which is also jointly typical with 
. Since there are exp{n(/5/(M/; t) + /(x; z) — 77)} sequences in the set, we can show along 
the same lines as in the proof of Lemma H] that can be decoded with high probability 
given (/CjZ"^, w^). The details will be omitted. 
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C. Converse 

Suppose there is a sequences of (n, A^) codes that achieves a secret key {k) rate of R, and 
[3 = N/n. Then from Fano's inequality, 

and from the secrecy constraint. 

-/(/c;z",u.^) 

n 

Combining these inequalities, we have that, 

ni?key < I{k- y^ - /(/c; z", i/i/^) + 2n5„ 
</(/c;y", 1/^ I z", i/^^) + 2ne„ 

< /i(y" I z") + h{v^ \ w^) - hiy"" I z", i/i/^, /c) - | y", z", u/^, /c) + 2ne„ 

< /i(y" I z") + I w^) - h{y'^ I z", w^, k, x") - | y", z", m/^, /c, ) + 2ne„ 
= /i(y" I z") + h{v^ \ w^) - h{y'' I z",x") - | y", z", i/i/^, k, ) + 2ne„ (98) 

n 

< 5^/(x,;y, I z,) + hiv"" I i^^) - /i(i/^|y", i/i/^, /c) + 2ri£„ (99) 
1=1 

< nl{x;y \ z) + h{v^ \ w^) - h{v^\y'\ , k) + 2nen (100) 

where the (l98l) follows from the fact that (i/i/^, /c) ^ (z", x") y", and (l99l) follows from the 
Markov condition z" — (y", m/", /c) ^ that holds for the degraded channel, while (1 1001) 
follows from the fact that /(x;y|z) is a concave function of (see e.g., [13, Appendix-I]) 
and we select px(-) = ^ Z]r=i ^^>^i(')- Now, let = (/(, u"_,„]^i/*~^,y"), J be a random variable 
uniformly distributed over the set [1, 2, . . . n] and t = (J, k, u'j_^_^v^~^, y") we have that 

N 

h{v^\y'', w^, k) = J2 h{vi\v'-\y'\ , k) 

i=l 
N 

>5^M^,|/-\y",w^,<i,/c) 

i=l 
N 

= ^/^(>/,|/-^y^v.„tyi^l,/c) (101) 

i=l 

= N ■ h{vj\t, wj) 

where we have used the fact that (i/i/'"^, n/j+i) — ^ (i/*"^,y", n/,, uj^^, k) i/j which can be 
verified as follows 

p {vi I Wi, w'-^, ",^i,y'', /() 

= ^p(i/, I Wi,Ui = u, 1/1/^1, y", /f) p (tVi = M I i/i/i,i/i/*-\i/i//^i,/-\ty,^i,y",/c) 

= X]^*^*^^ I = ("i = I i^i^ "i+i^y"^ ^) (102) 

=p(i/i I i/i/i,i/*"\ty,^i,y",/c) , 
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where (11021) follows from the fact that since the sequence is sampled i.i.d. , we have that 

and since u v ^ w, it follows that 

-> <„y", ^„ /c) ^ 

Since, vj and wj are both independent of J, we from (1 1001) that 

i?kcy < /(x;y|z) + + 2£„. 

Finally, using the steps between (l63l) - (|65l) as in the converse for the case when tv = 0, we 
have that 

I{x-y)>P{I{t-u)-I{t-v)), (103) 

which completes the proof. 

VIII. Public discussion channel 

We establish the upper bound on the secret key capacity in the presence of interactive 
communication over a public discussion channel. 
Proof: 

First from Fano's inequality we have the following, 

nR = H{k) (104) 
= H{k\l) + I{k-l) (105) 
< nsn + /(/c; /) (106) 

where the last inequality follows from Fano's inequality. Also from the secrecy constraint we 
have that 

-J(/c;0^^^z")<e„, 

n 

which results in the following 

nR<nen + I{k;l,tlj\(f)\z'') (107) 

< 2ne„ + J(/c; 0^ z") (108) 

< 2ne^ + I{m,, tv^; m^, y"|^^ 0^ z"), (109) 

where the last step follows from the data-processing inequality since k = K{m^, u^,ip'') and 
/ = L(m^,>/^,y'^,0'=). ■ 
Using the chain rule, we have that 

/(m.,u^m3„^^y"|V;^0^z") (110) 
= J(m„ u^; my, y", ^^ 0^ z") - u^; ^^ 0^ z") (11 1) 

n 

= I{m^, ty^; m,, u^, 0^-^) + + 

n 

- I{m^, u^; r-^) - 5^ + G„ (112) 
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where for each J = 1, 2, we define Fj = /(/TJx, u'^; yj, Zj|my, i/^,y-^ ^,z^ 0*^ ^, 
Gj = /(mx, u^;0i^.+i,...,0i^+i-i,V^i^+i,...,V'i,+i-i|my, v^,y^z^</)*^-\?/;^ and Fj = 



+ • • • ; Yij + i 

We now bound the expression in (II 121) . First note that 

/(m., u^; my, ct>'''^) - I{m^, tv^; 

< /(m., u^, my, 0*^-^) 

< /(m., u^; my, ^n-ilV^'^"', 0*^"') 
= /(m., u^my,^^|r-',r-') 

where the third and fifth step follow from the fact that = \I/.jj_.i(mx, u^, 0*^"^) and 

= $j^_i(my, 1/^, ^/''i^^). Recursively continuing we have that 

/(m., u^; my, ^/^ir"', r"') < /(^x, u^; ^y, u^) = /(u^; u^) = NI{u; v) (113) 

where we use the facts that ^ ^ ^ irty and that (u^, v^) are discrete and 
memoryless. 
Also note that 

F,-F, (114) 

= /(mx, u^; y„ | my, y^-\ z^-\ <f)^^-\^^^-') - I{m^, u^; z, |z^-\ 0^^-^) 

= i^(y„z,|my, v'',y^~\z^~\<P'^~\^P'^-') - H{y„z,\my, v\y^~\z^~\<P^^-\,lj^^-\ m^, u^) 

- H{z,\z^-\ 0*^-1) + H{z,\z^-\ (f)'^-\ mx, tv^) 

= if(y„z,|my, ^^,y^"\z-'-\0*^-\^*^-i) - H{y„z,\xj) - H{z,\z^~\ij'^-\<t>'^-^) + H{z,\x,) 

(115) 

<i/(y,|z^r-\0^^~')-^(y,|z,,x,) 

</(x,;y,|z,), (116) 

where (II 151) follows from the fact that Xj = Xj{m^, u^,ip^^~^) and that since the channel 
is memoryless (mx, rriy, , , (p^^~^ , , y^~^ , z^~^) — * xj iVj^Zj) holds. The last two 
steps follow from the fact that conditioning reduces entropy. 
Finally to upper bound Gj — Gj, 



Gj - G^ 



I{m^, u ; 0i^.+i, . . . , 0ij.+i--i, . . . , i)i^^,^i\my, v , y\ z\ 0'^-\ 

- /(mx, 0i^.+i, . . . , 0i,+i-i, . . . , ^i^.+i_i|z^ 0*^-\ ?/;*^-^) 

/(mx, u^; my, i/^, y^ 0^^.+!, . . . , 0i^.+,_i, ^Z-j^.+i, . . . , .+,_i|z^ 0*^-\ V*'"^) 

/(mx, tv^; my, \/^, y^'jz^ 0'^-\ ^^^"^) -/(mx, iv^;0i,-+i, . • • , 0i,.+i-i, t/^i^+i, • • • , i)^^^-x\z\ (j)'^-^^'^- 
I{m,, u^; my, v"" , y^(j)'^+^-\^/j^^+^-\ z^) - I{m,, u^; my, v"" , y^(f/^-\^/j'^-\ z^) 
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Furthermore since ^i^+i-i = $i,+i-i(mx, , ^) and ^i^+i-i = ^i^.+i_i(my, i/^, 0^^+^ ^) 
we have that 

/(m., u^; m,, z^) 

< I{m^, tv^; my, , ,4j,^^^_,\<f)^^+^-\^'^+^-\ z^) 
= I{m^, a^; niy, v^,y^, \(f)^^+^-\^^^+^-\ z^) 

Continuing this process we have that 

/(m., a^; my, / z^) < /(m., u^; m,, z^) 

and thus 

Gj-Gj<0. (117) 
Substituting (11131) . (11161) and (11171) into (11121) we have that 

n 

nR < ^I{xj-yj\zj) + NI{u; v) + 2n5„ (118) 

< maxn/(x;y|z) + iV/(u; \/) + 2n£:„ (119) 
thus yielding the stated upper bound. 

IX. Conclusions 

In this paper we introduced a secret-key agreement technique that harnesses uncertainties 
from both sources and channels. Applications of sensor networks and biometric systems 
motivated this setup. 

We first consider the case when the legitimate terminals observe a pair of correlated sources 
and communicate over a wiretap channel for generating secret keys. The secret-key capacity is 
bounded by establishing upper and lower bounds. The lower bound is established by providing 
a coding theorem that combines ideas from source and channel coding. Its optimality is 
established when the wiretap channel consists of parallel, independent and degraded channels. 
The lower bound in general involves us to operate at a point on the wiretap channel that 
balances the contribution of source and channel contributions and this illustrated for the 
Gaussian channels. 

In addition we also establish the capacity when the wiretapper has access to a source 
sequence which is a degraded version of the source sequence of the legitimate receiver. Fur- 
thermore the case when a public discussion channel is available for interactive communication 
is also studied and an upper bound on the secret-key capacity is provided. For the practically 
important case, when the wiretap channel consists of "independent noise" for the legitimate 
receiver and the discussion channel allows us to separately generate keys from source and 
channel components without loss of optimality. 

In terms of future work, there can be many fruitful avenues to explore for secret-key distilla- 
tion in a joint-source-channel setup. One can consider multi-user extensions of the secret-key 
generation problem along the lines of [6] and also consider more sophisticated channel models 
such as the compound wiretap channels, MIMO wiretap channels and wiretap channels with 
feedback and/or side information. Connections of this setup to wireless channels, biometric 
systems and other applications can also be interesting. 
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Appendix I 
Extension of Lemma [Uto general (a, b) 

We extend the coding theorem in section |IV] for Lemma [U to the case of general {a, b). 

We focus on the case when a = x. The general case then follows by further considering the 
auxiliary channel a ^ x, sampling the codewords from the typical set and then passing 
each symbol of a" through an auxiliary channel Px|a(-)- 

Our extension involves using a superposition code as discussed below. Let us define R^, = 
I{x;y\b) and i?b = I{t>]y)- Since 6 ^ x ^ y, we have that R\, + R.^ = I{x;y). We 
first generate a codebook with Ny, = exp (n(i?b — ^i)) sequences sampled uniformly 
from the set T^. For each sequence G Cb, we generate a codebook Ca(6") by selecting 
A^a = exp{n{I{x; y\b) — 5a)) sequences uniformly at random from the set T^i^{b^). 

Select 5a > and 5b > as arbitrary constants such that 5a. + 5h = 5, which satisfies (|23l) . 
Note that we have A^wz = ^a"^b- We define an encoding functions: $wz,i> '■ {1,2,..., A'b} 
Cb and $wz,a • {I, 2, • • • , A^a} — ^ (^aib'^) as a mapping from the messages to respective 
codewords in the codebooks. 

The construction of the Wyner-Ziv codebook and the secret-key codebook is via random 
partitioning along the lines in section IIV-AI — the constants Mwz and A'^wz are as given 
in (|25al) and (|25bl) respectively while 

MsK = exp {n{I{b; y) + I{x; z\b) - 5)) , (120a) 
A^SK = exp (n(/5J(t; v) + /(x;y|6) - /(x;z|ib) - 5)) . (120b) 

The encoding function is defined as follows: given a sequence , as in section IIV-BI a 
jointly typical sequence G T is selected and the bin index and secret-key are computed 
via the mappings $wz(t^) and ^sK{t^) respectively in Def. IH The bin index is split into 
two indices $a G {1,2,..., A'a} and $b G {1, • • • , A^b}, which form messages for the channel 
codebooks constructed above and the resulting sequence x" is transmitted. 

The decoder upon observing y" searches for sequences 6" G Cb and x" G Ca(/3") that are 
jointly typical i.e., (y",x", 6") G T^^^^^. By our choice of A^b and A'a this succeeds with 
high probability. It then reconstructs the bin index $wz and searches for a sequence E T 
that lies in this bin and is jointly typical with \/^. As in section HV-Ci this step succeeds with 
high probability. The secret-key is then computed as /c = $sK(f^)- 

We need to show the secrecy condition that 

ii/(/c|z") = {J(x; y\b) - J(x; z\b)} + /?/(t; v) + o,(l). (121) 
By expressing H{k\z'") as in ( |48] ) in section |lV-E.2| 

if(/c|z") = if(<l>wz|z") + if(t^|$wz) - i/(t^|/c, z"). (122) 
For the superposition codebook, since $wz is the transmitted message we have from [4] 

ii/(<Dwz|z") = /(x; y\b) - /(x; z\b) + o,(l), (123) 
and from (I27bl) in Lemma [3l 

i^i/(t^|$wz)=/(t;i/) + o,(l). (124) 

To show that 

-i/(t^|z",/c)=o,(l) (125) 
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we use a decoder analogous to that in the proof of Lemma |4] in Section ITV-Ei Upon observing 
z", the decoder searches for a sequence 6" G C\, that is jointly typical. This event succeeds 
with high probability since I{b;z) > I{b;y) = Ry,. Let the set of conditionally typical 
sequences x" be 

Jx = {j|x; G C^m, (x;,z") G T,^,,J. (126) 

The eavesdropper searches for all sequences gj^ such that $a(ffcj sk) ^ and ^h{tkj sk) = 
i Since the number of sequences gj^ is M^k = exp {n{I{x; z\b) + /(b; y) — 5)), along the 
lines of Lemma IH it follows that the codeword sequence is decoded with high probability. 
Note that (fml) follows from (fT22l) . (fT23l) . (fT24l) and (fT25]) . 



Appendix II 
Conditional Entropy Lemma 

Lemma 6: Suppose that the random variables a, b, and c are finite valued with a joint 
distribution Pa,b,ci ) that satisfies a ^ b ^ c. Suppose that a set Cc is selected by drawing 
exp(A^i?) sequences {cf} uniformly and at random from the set of typical sequences 
where R < H{c). Suppose that the pair of length- sequences (a^, b^) are drawn i.i.d. from 
the distribution t, and a sequence cf G Cc is selected uniformly at random from the set of 
all possible sequences such that (cf , b^) G Tj^^. Then for R > I{c; a), we have that 

liJ(cf |a^) = R- I{c- a) + o,(l), (127) 
where the term o^(l) vanishes to zero as — > oo and 77 ^ 0. 

Proof: From (|24c| ). for all pair of sequences (a^, b^), except a set whose probability is 
Ori{l), we have that (a^,6^) G Tj^^. For each such typical pair, since a ^ b ^ c and 
(6^, c/^) G T^^ from the Markov Lemma it follows that (a^, cf ) G Tj^,,. 

To establish (11271 ) it suffices to show that for all sequences G Tj^, except a set whose 
probability is at most o^(l) 

Pr(c^ = cf |a^ = a^) = exp(-Ar(/? - /(c; a) + o,,(l))). (128) 

The expression in (11271) then immediately follows by due to the continuity of the log function. 
To estabUsh (fT28l) . 



Pr(c^ = cf |a^ = a^) = '^^ ^. (129) 



j9(a^|c f)Pr(c ^ 



From property (I24bl) of typical sequences p{a^) = exp{—N{H{a) + o^(l))), p(a^|cf ) = 
exTp{—N(H(a\c) + o^(l))) and since the sequence is uniformly selected from 2"^ se- 
quences, we have that Pr(c^ = cf) = exp(— A^i?). Substituting these quantities in ( 11291 ) 
establishes (11281) . ■ 
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